Surgical Oncology 38 (2021) 101637

Available online 27 July 2021

0960-7404/© 2021 Published by Elsevier Ltd.

Performance of image guided navigation in laparoscopic liver surgery A

systematic review

C. Schneider a,*, M. Allam a,b, D. Stoyanov c,d, D.J. Hawkes d,e, K. Gurusamy a, B.R. Davidson a

a Department of Surgical Biotechnology, University College London, Pond Street, NW3 2QG, London, UK

b General surgery Department, Tanta University, Egypt

c Department of Computer Science, University College London, London, UK

d Centre for Medical Image Computing (CMIC), University College London, London, UK

e Wellcome / EPSRC Centre for Surgical and Interventional Sciences (WEISS), University College London, London, UK

A R T I C L E I N F O

Keywords:

Laparoscopic liver surgery

Laparoscopic liver resection

Robotic liver surgery

Image guided surgery

Computer assisted surgery

Computer assisted navigation

Augmented reality

Machine vision

A B S T R A C T

Background: Compared to open surgery, minimally invasive liver resection has improved short term outcomes. It

is however technically more challenging. Navigated image guidance systems (IGS) are being developed to

overcome these challenges. The aim of this systematic review is to provide an overview of their current capa-

bilities and limitations.

Methods: Medline, Embase and Cochrane databases were searched using free text terms and corresponding

controlled vocabulary. Titles and abstracts of retrieved articles were screened for inclusion criteria. Due to the

heterogeneity of the retrieved data it was not possible to conduct a meta-analysis. Therefore results are presented

in tabulated and narrative format.

Results: Out of 2015 articles, 17 pre-clinical and 33 clinical papers met inclusion criteria. Data from 24 articles

that reported on accuracy indicates that in recent years navigation accuracy has been in the range of 815 mm.

Due to discrepancies in evaluation methods it is difficult to compare accuracy metrics between different systems.

Surgeon feedback suggests that current state of the art IGS may be useful as a supplementary navigation tool,

especially in small liver lesions that are difficult to locate. They are however not able to reliably localise all

relevant anatomical structures. Only one article investigated IGS impact on clinical outcomes.

Conclusions: Further improvements in navigation accuracy are needed to enable reliable visualisation of tumour

margins with the precision required for oncological resections. To enhance comparability between different IGS

it is crucial to find a consensus on the assessment of navigation accuracy as a minimum reporting standard.

1. Introduction

Laparoscopic liver resection (LLR) has benefits over open resection in

terms of improved patient recovery, better cosmesis, shorter length of

hospital stay and reduced morbidity [15]. Unfortunately complex LLR

such

as

major

hepatectomies

and

segmental

resections

in

superior-posterior segments are technically challenging and have

therefore seen a slow uptake by the surgical community [1,3,6].

A number of factors make LLR technically more challenging than

open resection. The inability to palpate the liver parenchyma makes it

difficult to detect small liver lesions which has caused concerns about

oncological clearance. Because of the livers complex three-dimensional

(3D) structure that is derived from its vascular anatomy, it can be

challenging to find and maintain the correct anatomical orientation

within two-dimensional (2D) laparoscopic view which does not provide

depth perception. Poor orientation may lead to incomplete oncological

resection and inadvertent vascular or biliary injury [3,710].

Laparoscopic ultrasound (LUS) may be used prior to parenchymal

transection to identify liver lesions and delineate the hepatic vasculature

[1115]. Once transection has started, however, use of LUS is

demanding because it only provides 2D images which are difficult to

interpret in conjunction with the orientation of the laparoscopic camera.

An additional limitation of LUS is that its diagnostic accuracy is

decreased in the presence of liver cirrhosis, small- or vanishing liver

lesions [8,1619].

Robotic assisted liver resection has been introduced to overcome the

innate limitations of laparoscopic instruments. Surgical dexterity is

improved by utilisation of endo-wristed instruments with 7 of freedom

whereas routine use of stereoscopic laparoscopy enhances depth

* Corresponding author.

E-mail address: crispin.schneider.13@ucl.ac.uk (C. Schneider).

Contents lists available at ScienceDirect

Surgical Oncology

journal homepage: www.elsevier.com/locate/suronc

https://doi.org/10.1016/j.suronc.2021.101637

Received 12 April 2021; Received in revised form 4 July 2021; Accepted 24 July 2021

Surgical Oncology 38 (2021) 101637

2

perception [20]. Similar to LLR however, it is not possible to palpate the

liver and intraoperative interpretation of the 3D anatomical situation is

taxing.

To address these issues image guidance navigation systems (IGS) that

enable intraoperative visualisation of the liver anatomy are being

developed. IGS aim to display anatomical data, spatially correlated to

the operative site, often in the form of 3D models that are created from

cross-sectional imaging. Use of IGS in LLR is particularly appealing

because the display of the highly variable vascular and tumour anatomy

may aid in identifying tumour margins as well as blood vessels and bile

ducts [21,22]. Although IGS are currently widely used in neurosurgery,

orthopaedic surgery and otolaryngology, its evolution in LLR has been

slow [23]. The main obstacles preventing meaningful implementation of

this technology are the mobility of abdominal organs, lack of fixed bony

landmarks for orientation and organ motion secondary to diaphragmatic

and cardiac movement [8,23,24]. Further issues are the paucity of liver

surface features and significant soft tissue deformation due to the

increased intra-abdominal pressure from the pneumoperitoneum and

surgical manipulation [24].

Because of the complexity of the technical challenges a number of

IGS technologies have been developed. These can be broadly categorised

according to the underlying imaging modality into video, ultrasound,

computer tomography (CT) and magnetic resonance imaging (MRI)

-based systems. The aim of this systematic review is to provide a

comprehensive overview of the potential benefits and limitations of IGS

in minimally invasive liver surgery.

2. Methods

A systematic literature search that included the free text and corre-

sponding controlled vocabulary terms for liverand laparoscopy

combined with those for computer vision terms (e.g. machine vision,

augmented reality), or image guided surgerywas performed using the

Medline, Embase and Cochrane databases. A detailed description of the

search strategy is stated in Appendix 1. To complement the initial

search, each Medline search term indexed under Diagnostic Techniques

and Procedureswas screened for relevant image guidance modalities

and included as a separate search term if appropriate.

Full text articles, conference -proceedings and -abstracts describing

in-vivo pre-clinical studies or clinical research on image guidance sys-

tems in minimally invasive liver -resection or -ablation were retrieved.

No backward time restriction was applied to the search and articles

published up to the December 31, 2020 were included.

Exclusion criteria were image guidance for radiotherapy purposes,

ex-vivo research, non-registered image guidance (e.g. preoperative

planning) or non-primary research. No articles were excluded based on

language. Articles reporting on imaging in open liver resection or

laparoscopic cholecystectomy were also excluded. To ensure mid-term

clinical relevance, this review focuses exclusively on in vivo studies.

Systems that do not provide navigation (i.e. lack spatial correlation) are

not reviewed. Screening of the titles and abstracts of retrieved references

was independently carried out by two authors (CS & MA). In case of

disagreement a discussion took place and if the disagreement persisted,

the final decision about inclusion was made by the senior author (BD).

Full texts for eligible articles were retrieved and read. A narrative

summary of the findings is given in table and prose form. Where

possible, system performance is quantified with objective data such as

navigation accuracy and setup time. As the methodology used in the

studies varied significantly no quantitative analysis or meta-analysis

could be conducted.

2.1. General aspects of image guidance in laparoscopic surgery

Most IGS are based on three key components or processes which are:

1) 3D modelling - to create a virtual representation of patient anatomy

2) registration and tracking - to align virtualand real anatomy and 3)

Visualisation - to make the information interpretable. 3D modelling is

facilitated by processing volumetric data from CT or MRI scans. For LUS,

CT and MRI -IGS, 3D models are not mandatory since these modalities

have the capability to directly visualise liver anatomy during surgery.

Registration is the technically most challenging step and is thought

to have the greatest impact on navigation accuracy (i.e. how precisely

imaging reflects anatomy). To facilitate registration it is necessary to

obtain biometrical features of the patients liver that can be aligned with

corresponding features on the 3D model. These features may consist of

only a few anatomical landmarks [17] or conversely they may incor-

porate a detailed geometrical liver surface representation [8]. In its most

simple form registration can be carried out manually where the surgeon

aligns 3D model and laparoscopic view [2529]. Some groups advocate

outlining the liver landmarks with a tracked stylus. Subsequent regis-

tration is achieved by computing the minimum distance between in vivo

and virtual landmarks [8,30]. Laser range scanning may offer an alter-

native method for obtaining biometrical liver data [31].

More recently, semi-automatic registration methods have been

popularised. Most commonly a technique called stereoscopic surface

reconstruction (SSR) that requires a 3D laparoscope also known as a

stereoscope is employed. The right and left video channels of the ste-

reoscope triangulate points on the liver surface (Fig. 1) which are

Abbreviations

AR

augmented reality

CBCT

Cone beam computer tomography

CNN

convolutional neural network

CRLM

colorectal liver metastasis

CT

Computer tomography

FPS

frames per second

IGS

Image guidance system

LLR

Laparoscopic liver resection

LUS

Laparoscopic ultrasound

MRI

Magnetic resonance imaging

SLAM

Simultaneous localisation and mapping

SSR

stereoscopic surface reconstruction

TRE

Target registration error

US

Ultrasound

Fig. 1. Graphic illustrating the concept of SSR. On the left is a 3D laparoscopic

camera with a right and left video channel pointing towards the liver surface on

the right. Viewing the same point through two different spatially fixed video

channels allows calculation of the point-to-camera distance. Reprinted with

permission from Springer Nature [32].

C. Schneider et al.

Surgical Oncology 38 (2021) 101637

3

subsequently amalgamated into a point cloud that is essentially a 3D

points representation of the liver surface. Thereafter a process called ICP

matching is used to align 3D model and point cloud to complete regis-

tration [32].

Tracking provides positional information which enables spatial

correlation between laparoscope, patient anatomy and surgical in-

struments. Optical tracking is the most common method which employs

reflective infrared markers that are attached to instruments [8,33,34].

The position of these markers is recorded by an optical tracking camera

that requires a direct line of sight. This limitation can be avoided by

using electromagnetic (EM) tracking which utilises phase changes

within an EM field to determine positional changes. Calibration is the

process that informs the fixed spatial relationship between tracking

markers and camera optics. Novel concepts such as iterative closest

point (ICP)- and simultaneous localisation and mapping (SLAM)-

tracking are further detailed below.

Earlier systems utilised separate screens to show laparoscopic view

and 3D model next to each other. More recently augmented reality (AR)

displays have been increasingly employed. The advantage of AR is that

patient anatomy and 3D model are visualised on the same screen in an

overlay fashion (Fig. 2). AR is thought to render image interpretation

more intuitive and an additional advantage is that surgical instruments

do not require tracking because they are directly observed within the AR

environment.

Navigation accuracy is often expressed as target registration error

(TRE) which measures how accurately image guidance reflects the

anatomical situation. As a simplification it can be regarded as the sum of

registration- and tracking-error, with the former being the main

contributor to the overall error. Because TRE evaluation is not stand-

ardised, care has to be taken when comparing different IGS [8,24,25]. In

general TRE is calculated by measuring the distance between corre-

sponding landmarks on the 3D model and the patients anatomy.

3. Results

The initial search identified 2015 articles (Fig. 3). Following

screening of titles and abstracts, 1953 articles were excluded. After re-

view of full texts a further 12 articles were excluded, because they either

did not involve in vivo studies (n = 4), studied only cholecystectomy (n

= 1), did not include navigation(n = 3) or were only based on open

surgery (n = 4). Eventually 50 eligible articles, 17 based on preclinical

and 33 based on clinical research were eligible for inclusion. Pre-clinical

research was exclusively conducted on pigs. Information on methodol-

ogy, number of test subjects, key findings and limitations were retrieved

Fig. 2. AR visualisation showing the 3D model overlayed onto the operative site. The liver surface is not displayed to allow a clearer view of blood vessels and bile

ducts (hepatic veinsblue; portal veinspurple; arteriesred, bile ducts & gallbladdergreen). (original images by Ref. [75] licensed under CC-BY 4.0). . (For

interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

Fig. 3. Flowchart for selection of articles.

C. Schneider et al.

Surgical Oncology 38 (2021) 101637

4

and summarised in text and table format. To provide an introduction to

the topic and standardise terminology, the results section begins with a

brief description of the key principles underlying IGS and a summary of

relevant findings.

3.1. Video IGS

The first article on Video-IGS published in 2006, investigated laser

range scanning based surface reconstruction in a porcine model [31].

Since this publication there have been no new in vivo studies on this

registration approach and in general most groups prefer to utilise

manual registration with a tracked stylus or user manipulated overlay.

Projecting 3D models externally onto a patients skin may aid laparo-

scopic port placement but visualisation can be altered by ports, in-

struments, and the uneven outline of the abdomen [35].

Currently AR is the most popular visualisation method because, as

demonstrated in a porcine IGS study [36], it is thought to facilitate

mental integration between image guidance data and operative site. The

first clinical report on AR visualisation in LLR was published in 2011

[37]. AR is also a natural fit for robotic assisted liver resection since it

utilises the inherent stereoscopic view of the DaVinci[17] console.

3.1.1. Surface reconstruction

Surface reconstruction describes the acquisition of biometric liver

surface characteristics or in other words reading the liver surface.

These characteristics can be used for semi-automatic registration but

also to provide data streams to drive modelling of liver deformation (see

below). It has been demonstrated in two porcine studies that semi-

automatic registration is advantageous because it is less time

consuming than manual registration and not influenced by user

dependent registration errors(38,39).

Up to date SSR is the most widely researched surface reconstruction

method. The first in vivo evaluation was published in 2015 on a porcine

model. Using a non-deformable 3D liver model the authors achieved a

TRE10 mm. It has been postulated that implementation of a deform-

able 3D model could improve the TRE to approximately 34 mm [25].

The application of SSR in humans has been more difficult. Some of the

proposed methods to overcome this issue have been the use of deep

learning to automatically segment (i.e. distinguish) the liver from sur-

rounding organs [40] and the application of a scoring method to identify

the optimal laparoscope position for SSR [29].

SSR can also facilitate tracking without the need for dedicated

tracking equipment. One group proposed the use of ICP tracking, a

method that utilises changes in liver surface biometry to infer laparo-

scope position. Studied in pre-clinical experiments this approach

worked in real-time but navigation accuracy was inferior to that of op-

tical tracking [33].

A potential alternative to SSR is SLAM which is a concept in

computational geometry that enables updating of a map (e.g. liver sur-

face) in an unknown environment while simultaneously tracking objects

[32]. Using a standard monocular laparoscope, it has been demonstrated

in a pre-clinical [41] and a clinical study [42] that SLAM has the po-

tential to enable synchronous tracking and liver surface reconstruction.

3.1.2. Tissue deformation

Most IGS employ a rigid 3D model that cannot adjust shape or po-

sition to reflect physical forces (e.g. respiratory motion, surgical

manipulation) exerted onto the liver. Based on results from porcine

experiments, it has been postulated that deformable liver modelling is

crucial in achieving navigation accuracies of <4 mm [25] and hence

many researchers perceive this to be the holy grail of navigated image

guidance.

The majority of publications are based on retrospective patient video

data [24,4345] whereas only some groups have attempted intra-

operative evaluation in porcine [46,47] and human [48] studies.

Various models based on complex problem-solving principles in maths

and physics have been postulated but a detailed methodological

description goes beyond the scope of this review.

One of the main obstacles to clinical translation is the substantial

computational expense (i.e. processing power demand), which makes it

challenging to simulate deformable modelling in real-time. Generally,

solutions can be categorised into biomechanical models and data driven

models. The most popular biomechanical solution which has been suc-

cessfully employed in patients, is the finite element method which uti-

lises an organ mesh to represent tissue deformation [43,49]. Potentially

less computationally expensive are data driven models which can be

trained by observing laparoscopic video or synthetic simulations. These

models utilise convolutional neural networks (CNN), a form of machine

learning, which can use graphic processing units to drastically increase

computing speed. It has been suggested that this advantage should

enable real-time functionality in a clinical setting [44]. To the best of our

knowledge however neither biomechanical nor data driven -models

have been able to reliably simulate liver deformation in porcine [47] or

clinical

[43,44]

studies.

In

summary,

AR

visualisation

and

semi-automatic registration are gaining popularity and have the po-

tential to make Video-IGS easier to use. Fundamental improvements to

navigation accuracy will probably depend on the development of reli-

able real-time tissue deformation.

3.2. Laparoscopic ultrasound IGS

One of the greatest obstacles in employing LUS is the difficulty of

mentally integrating 2D US and laparoscopic images. Therefore the

main focus of research has been on developing IGS that integrate LUS

information into the intraoperative environment. The majority of LUS-

IGS utilise B-mode US images as the primary source of visualisation

[40,41] and hence integration of a 3D model is not mandatory. The first

report on LUS-IGS was published in 2014 by a group that overlayed LUS

images onto a 3D laparoscopic video feed in a porcine model. The au-

thors stated that their system facilitated intuitive visualisation of

sub-surface structures [36]. Optical tracking as utilised by this group

cannot be combined with flexible LUS probes since changing the angle of

the probe head is not reflected by the position of the optical tracker. To

address this problem an IGS employing EM tracking markers at the tip of

the LUS probe was developed and evaluated in a pre-clinical study [50].

Another group demonstrated in a clinical setting that LUS images may

also be co-registered with CT images (i.e. correlating LUS images with

spatial location on cross-sectional imaging) to aid in their simultaneous

interpretation(51). It has been shown that LUS-IGS may aid laparoscopic

liver ablation by enabling stereoscopic visualisation of probe trajectory

and tumour position. In a series of 13 patients complete ablation was

achieved in 12 cases [52]. Rather than using LUS for visualisation, one

group demonstrated how it can be utilised for registration instead. Blood

vessel centrelines were acquired with EM tracked LUS in a porcine

model and this data enabled reconstruction of blood vessel anatomy

which subsequently facilitated registration to the corresponding blood

vessels on the 3D model. This approach also enabled integration of LUS

images within the 3D model [53] (Fig. 4). In summary, data so far

suggests that LUS-IGS seem to be particularly useful when co-registered

with CT images or a 3D model. EM tracking is becoming increasingly

popular since it is currently the only viable solution for tracking flexible

LUS probes.

3.3. Computer tomography IGS

CT-IGS have the capacity to acquire volumetric anatomical data (e.g.

liver shape) during surgery. This can then be used for direct visualisation

of liver anatomy or for registration. A crucial step for the advent of CT-

IGS has been an increased availability of cone beam CT (CBCT) within

operating theatres. The first publication on this topic in 2008 reported

the use of optically tracked CBCT during porcine laparoscopy. Regis-

tration of non-contrast and contrast enhanced CBCT was facilitated by

C. Schneider et al.

Surgical Oncology 38 (2021) 101637

5

attaching fiducials to either the skin or the liver surface, respectively.

Following AR visualisation, navigation accuracy was app. 11 mm [54].

Two years later an IGS based on either intermittent or continuous low

dose, non-contrast CT was developed and evaluated in a preclinical

experiment. The low radiation dose of 25 mA enabled regular

re-registration to adapt the 3D model to intraoperative liver deformation

which resulted in a TRE of 1.45 mm. One-off rather than repeat regis-

tration was also explored but this resulted in decreased navigation ac-

curacy since adjustment to liver deformation was not feasible. Major

limitations were increased radiation exposure when using continuous

CT and the requirement for a multi-slice CT scanner within the operating

theatre [55]. Up to date, there has only been one report on CT-IGS

application in a patient. In this report, biometric liver data was ob-

tained by intraoperative CBCT to facilitate registration. Since the

tumour was only visible on MRI, a preoperative MRI was used to process

the 3D liver model. Intraoperative fluoroscopy enabled correlation be-

tween 3D model and surgical instruments [56]. In summary, CT-IGS

technology is a precise registration tool but radiation exposure is high

if it is used for intraoperative cross-sectional imaging.

3.4. Magnetic resonance imaging IGS

MRI guided liver ablation and surgery was made possible by the

invention of the open plane MRI scanner which in contrast to conven-

tional MRI scanners does not completely surround the patient and hence

allows access to conduct procedures. In 2009 a group explored the use of

open plane MRI in a porcine model of LLR. They determined that a T2

weighted sequence with fast spin echo provided the best image quality

while offering an acceptable image acquisition time. An electromag-

netically shielded control room contained all non-MRI compatible

equipment. Within the MR field surgeons used non-ferromagnetic

laparoscopic ports in conjunction with a Nd:YAG laser which enabled

tissue dissection and coagulation. The Nd:YAG titanium manufactured

laser handle was marked with Gadolinium to aid its localisation in MR

images [57]. The only other MRI-IGS study evaluated laparoscopic mi-

crowave ablation. Surgical instruments were constructed from weakly

ferro-magnetic materials. The authors described successful ablation in 6

patients [58]. No 3D models were used in either of these works since

MRI-IGS enabled direct correlation between instruments and liver

anatomy (Fig. 5). In summary, MRI-IGS offers outstanding imaging

quality compared to other IGS modalities but has restrictions in terms of

operating room setup and instrument compatibility.

3.5. Data summary tables

For a table summary of included preclinical and clinical articles

please see (Table 1) and (Table 2), respectively.

4. Discussion

This review has highlighted the current state of the art in navigated

image guidance for minimally invasive liver surgery. The majority of

publications are less than 10 years old which indicates that this tech-

nology is evolving rapidly. IGS have been evaluated in clinical scenarios

right from the inception of this technology, a fact that is reflected by the

large proportion of clinical articles in this review. Most studies were of

Fig. 4. LUS images can be integrated into an

AR 3D model to enhance spatial correlation.

A) LUS probe (in black) examining porcine

liver. B) LUS image (monochrome square

image) is integrated into a 3D porcine liver

model. The position and content of the LUS

image changes when the LUS probe is

moved. Therefore there is spatial correlation

of intrahepatic structures (e.g. blood vessel

arrow) on LUS image and 3D model. The

liver borders are outlined in grey, hepatic

veins are blue and portal veins are purple.

Tumour locations are shown as yellow le-

sions. (original images by Ref. [53] licensed

under CC-BY 4.0). . (For interpretation of the

references to colour in this figure legend, the

reader is referred to the Web version of this

article.)

Fig. 5. A) Open plane MRI configuration restricts the surgeons range of movement. Laparoscopic and MRI images can be visualised by non-ferromagnetic screens

placed at the rear opening of the scanner B) Direct intraoperative visualisation of spatial relationship between the surgeons fingers (arrows) and the liver (L). Liver

vessels can be seen as dark circles within the parenchym. reprinted with permission from Springer Nature [85].

C. Schneider et al.

Surgical Oncology 38 (2021) 101637

6

Table 1

Pre-clinical studies

Author &

Journal &

Date and Country

Imaging

modality & No.

of subjects

Study design

& Type of

surgery

Methodology

Important findings

Important limitations

Hayashibe et al. [31]

Medical Image Analysis

August 2006, Japan

Video

n = 1

Exploratory

Laparoscopy

- Registration with laser

surface scanning

- Allows reconstruction of

biometrical liver surface data

in real time.

- Prevents collision of robotic

instruments.

- No registration or

visualisation

demonstrated.

- One subject only.

Konishi et al. [59]

IJCARS

June 2007, Japan

LUS

n = 12

Exploratory

Lap. ablation

- Co-registration of 3D LUS

and video.

- Optical tracking for rigid

instruments.

- EM tracking with

magnetic distortion

correction for flexible

instruments.

- Magnetic distortion

correction improved

navigation accuracy from

17.2 mm to 1.96 mm.

- LUS scanning time app. 30s.

- Time to generate images app.

3 min.

- Lacks comparison of

optical and EM

tracking.

Feuerstein et al. [54]

IEEE Transactions on Medical Imaging

March 2008, Germany

CT-AR

n = 2

Exploratory

LLR

- CBCT used to create 3D

model.

- Display at expiration

only to account for

respiratory motion.

- Navigation accuracy ¼

11.05 ± 4.03 mm.

- Visualisation of major liver

vessels aided in laparoscopic

port placement.

- Respiratory motion increased

TRE by app. 10 mm.

- Unable to visualise

peripheral liver

vessels.

- 3D model lacks detail

Chopra et al. [57]

European Radiology September 2009,

Germany

MRI

n = 2

Exploratory

LLR

- Suitability of different

MR sequences evaluated.

- Development of MR-

compatible theatre setup.

- Optimal MRI sequence is T2

fast spin echo.

- Nitinol built laparoscope is

MR compatible.

- Tissue dissection with 1064-

nm Nd:YAG laser is feasible

and MR-compatible.

- No AR visualisation.

Shekhar et al. [55]

Surgical Endoscopy

August 2010, USA

CT-AR

n = 6

Exploratory

Laparoscopy

- Intraoperative multi-slice

CT (not CBCT).

- Registration with

continuous or non-

continuous low dose non-

contrast CT.

- Navigation accuracy ¼

1.45 mm (low dose) vs.

1.47 mm (high dose).

- Low dose CT reduces

radiation exposure eight fold.

- Continuous scanning enabled

registration updates.

- High radiation

exposure with

continuous CT

compared to CBCT.

Kang et al. [36]

Surgical Endoscopy

July 2014, USA

LUS-AR

n = 2

Exploratory

Laparoscopy

- Overlay of LUS images

onto 3D laparoscopic

view.

- Successful registration of

intrahepatic structures

- Dark tissues (e.g. kidney)

provide better contrast for

overlaying LUS images.

- Feasibility only

demonstrated with

rigid LUS probe.

Thompson et al. [25]

SPIE proceedings

March 2015, UK

Video-AR

(SmartLiver) n

= 5

Exploratory

LLR

- Semi-automatic

registration with SSR

- Accuracy comparison

between rigid and

deformable 3D models.

- Navigation accuracy

app.10 mm.

- Successful registration n = 3/

5.

- Extensive liver deformation

caused failure of SSR.

- Comparison rigid and

deformable 3D models

based on simulation

only.

Reichard et al. [33]

Journal of Medical Imaging

October 2015, Germany

Video-AR

n = 1

Exploratory

Laparoscopy

- SSR registration.

- Comparison of optical

and ICP tracking.

- Navigation accuracy ¼ 13

mm.

- Best accuracy with combined

ICP & optical tracking.

- ICP tracking is more accurate

with HD laparoscope.

- Maximum frame rate 4/s.

- ICP tracking not

working in real-time.

- One subject only.

Song et al. [53]

IJCARS

December 2015, UK

LUS-AR

(SmartLiver) n

= 2

Exploratory

Laparoscopy

- Registration to vascular

landmarks with EM

tracked LUS

- Navigation accuracy ¼

3.74.5 mm.

- Accuracy better in proximity

to landmarks.

- LUS images integrated into

3D model.

- No comparison of SSR

vs. LUS registration.

Reichard et al. [47]

IJCARS

July 2017, Germany

Video-AR

n = 1

Exploratory

Laparoscopy

- Semi-automatic

registration with SSR.

- Deformable,

biomechanical 3D liver

model.

- Demonstrated real-time

registration and deformation

on porcine spleen.

- In-vivo data on spleen

only.

- No in vivo accuracy

data.

- One subject only.

Ramalhinho et al. [60]

IJCARS

August 2018, UK

LUS-AR

(SmartLiver) n

= 1

Exploratory

Laparoscopy

- LUS registration as in

Ref. [53].

- Computer simulation to

determine optimal LUS

probe positions for

registration.

- Navigation accuracy ¼

10.416.3 mm.

- Higher vascular density in

central liver segments

improves registration.

- Re-evaluation of data

from Ref. [53] but

reports new findings.

- One subject only.

Lau et al. [50]

J Laparoendosc Adv Surg

January 2019, USA

LUS-AR

n = 1

Exploratory

LLR

- EM tracked LUS.

- LUS images overlayed

onto laparoscopic view.

- LLR with AR 7 min. vs. 3 min.

without AR.

- No accuracy data.

- One subject only.

(continued on next page)

C. Schneider et al.

Surgical Oncology 38 (2021) 101637

7

an exploratory nature and were not designed to demonstrate clinical

benefits. This is perhaps unsurprising since at this development stage the

research focus has been on innovation rather than clinical validation.

Twenty-four articles in this review published quantitative data on

navigation accuracy. The methodology of navigation accuracy assess-

ment varies between research groups and therefore it is difficult to

compare results directly [61]. Despite these disparities there appears to

be some evidence that studies using retrospective registration [24,43]

and studies with only one subject [29,66] tend to report better naviga-

tion accuracy which may point towards associated bias. Recently, the

proportion of publications stating accuracy data is increasing, which

perhaps reflects the recognition by scientists that quantifiable data is

paramount to advance the field (Fig. 6).

The advent of AR has been an important development. Whereas

earlier systems relied on two separate screens, AR offers more intuitive

visualisation. Utilisation of AR may cause information overload [70]

which can be addressed by allowing surgeons to switch between full AR,

limited AR (e.g. area of interest, limited opacity) and no AR [70,74].

Enhanced rendering has been proposed as another potential solution

[78]. This technology employs a variety of graphics processing methods

such as plane clipping, distance fogging and shape outlining to focus the

surgeons attention on relevant anatomical details (Fig. 7).

Judging by the number of publications, Video-IGS have seen the

most attention by the research community. This popularity can perhaps

be explained by advantages such as user friendliness, low costs, porta-

bility, high image acquisition speed, and compatibility with existing

surgical equipment [6,23,24]. Its main disadvantage is a lack of depth

penetration which means that the position of deep lying structures can

only be inferred from a 3D model whereas LUS-, CT- and MRI-IGS may

offer direct visualisation of deep structures. Attempts at developing

deformable 3D liver models have been promising [24,44,47] but so far

no group was able to demonstrate real-time functionality during sur-

gery. A previous study estimated that under optimal circumstances a

rigid 3D model could yield a TRE of 810 mm. One-off deformation to

adapt to relatively constant changes in liver shape (e.g. after liver

mobilisation) may achieve TREs of 56 mm whereas real-time soft tis-

sue deformation may further improve the TRE to 23 mm [25]. Up to

date, deformation research in LLR has not formally addressed the impact

of liver transection. In open liver surgery it was observed that liver

transection causes up to 8.7 mm displacement of intrahepatic blood

vessels [79]. How this phenomenon will be incorporated into deform-

able 3D liver models for LLR remains to be seen. That deformable 3D

models have so far remained elusive, can perhaps explain why some data

points towards better navigation accuracy for CT and LUS -IGS [53,55,

56,59].

SSR which requires expensive 3D laparoscopes is currently the most

Table 1 (continued)

Author &

Journal &

Date and Country

Imaging

modality & No.

of subjects

Study design

& Type of

surgery

Methodology

Important findings

Important limitations

- Comparing liver

resection margins, AR vs.

no ARAR.

- Clear resection margins in

both groups.

Modrzejewski et al. [46]

IJCARS

April 2019, France

Video-AR

n = 1

Exploratory

Laparoscopy

- Semi-automatic

registration with SSR.

- Deformable,

biomechanical 3D liver

model.

- Various dataset of liver

deformation recorded for

public use.

- Navigation accuracy ¼ 20

mm (intrahepatic) vs. 15

mm (liver surface).

- Self-collision restraint of

deformable 3D model

improved navigation

accuracy by 12 mm.

- SSR methodology not

described in detail.

- One subject only.

Luo et al. [61]

Computer Methods and Programs in

Biomedicine

September 2019, China

Video-AR

n = 5

Exploratory

LLR

- Semi-automatic

registration with SSR.

- 3D modelling and

registration with

convolutional neural

networks.

- Liver surface fiducials to

aid registration.

- Navigation accuracy ¼ 8.7

± 2.4 mm -Liver surface

reconstruction and

registration in app. 3 min.

- Frame rate 1012 fps ex-vivo.

- Review of different accuracy

evaluation methods.

- Navigation not in real-

time.

- In vivo frame rate not

stated.

- Requires

intraoperative CT.

Teatini et al. [38]

Scientific Reports

December 2019, Norway

Video-AR

n = 4

Exploratory

Laparoscopy

- Manual registration.

- Creation and comparison

of pre- (multislice CT)

and intra-operative

(CBCT) 3D models.

- Evaluation fiducials vs.

user-defined landmarks.

- Navigation accuracy ¼

19.04 mm (intraoperative

3D model) vs. 38.37 mm

(preoperative 3D model).

- Landmark dependent error

20.3 mm (manual selection)

vs. 14.38 mm (fiducial).

- Accuracy improved with

minimum 45 landmarks.

- Fiducial results only

for three subjects.

- Unclear how visible

diathermy marking is

on CT liver.

Teatini et al. [39]

Min Invasive Ther

Jan 2020, Norway

Video-AR

n = 1

Exploratory

Laparoscopy

- Comparison of manual

registrations by different

surgeons.

- Evaluating impact of

sampling error on

accuracy.

- Navigation accuracy 13.37

± 6.25 mm

- Different accuracy results

between surgeons (p =

0.00045).

- Only one subject

- Usage of different

accuracy metrics is

confusing.

Liu et al. [62]

IJCARS

May 2020, USA

LUS

n = 1

Exploratory

Lap. ablation

- EM tracked LUS

- LUS images showing

needle trajectory

overlayed onto

laparoscopic view.

- IGS feasibility demonstrated.

- Artificial tumours

successfully targeted.

- Comparison of AR vs.

LUS guided needle

placement ex vivo

only.

Table 1. Summary of included preclinical articles. Navigation accuracy data is highlighted in bold. Journal name abbreviations: IJCARS - International Journal of

Computer Assisted Radiology and Surgery; J Laparoendosc Adv Surg - Journal of Laparoendoscopic & Advanced Surgical Techniques; Min Invasive Ther - Minimally

Invasive Therapy & Allied Technologies.

C. Schneider et al.

Surgical Oncology 38 (2021) 101637

8

Table 2

Clinical studies.

Author &

Journal &

Date and Country

Imaging modality

& No. of subjects

Study design

& Type of

surgery

Methodology

Important findings

Important limitations

Volont´e et al. [35]

J Hepatobil Pancreat

Sci

April 2011,

Switzerland

Video-AR (OsiriX)

n = not stated

Exploratory

Robotic

- Manual registration to external

landmarks.

- Projection of 3D model on patient

skin.

- External projection aided in

laparoscopic port placement.

- 3D model distorted by

instruments and ports.

- No accuracy data.

Nicolau et al. [37]

Surgical Oncology

September 2011,

France

Video-AR

n = 5

Exploratory

LLR

- Manual registration.

- Estimated portal vein position AR

vs. surgeon assessment vs. LUS

(control).

- Registration more precise with

small field of view.

- Repeat registration if field of view

changes.

- AR superior to surgeon assessment

in 2/5 cases.

- No accuracy data.

- IGS technology not

described.

Kingham et al. [8]

Journal of

gastrointestinal

surgery

July 2013, USA

Video (Explorer)

n = 32

Exploratory

Laparoscopy

- Manual registration and additional

laser surface scanning registration

in some open cases.

- Comparison open surgery vs.

laparoscopy at 7mmhg & 14

mmHg.

- Navigation accuracy ¼ 4.9 ± 1.3

mm (laparoscopic at 14 mmHg)

vs. 5.4 ± 2.1 mm (open).

- Accuracy comparable at 7 mmHg

vs. 14 mmHg.

- Registration time app. 3min.

- No performance metrics

for laparoscopic group

- No surgeon feedback.

Buchs et al. [17]

J Surg Res

October 2013,

Switzerland

Video (CAS-One

Surgery) n = 2

Exploratory

Robotic

- Manual registration.

- AR integrated into robotic console.

- IGS useful for localising lesions.

- Potentially faster manual

registration due to robotic tremor

elimination.

- No accuracy data.

Kenngott et al. [56]

Surgical Endoscopy

March 2014, Germany

CT-AR

n = 1

Exploratory

LLR

- CBCT registration using liver

volume reconstruction.

- 3D model constructed from MRI

since tumour not visible on CT.

- Feasible to determine optimal liver

transection plane.

- No in vivo accuracy data.

- No respiratory gating.

- One subject only.

Satou et al. [26]

Hepatology Int.

March 2014, Japan

Video-AR n = 7

Exploratory

LLR

- Manual registration.

- Intraoperative tumour location

correlated with AR.

- No accuracy data.

- Technology not

described.

Hammill et al. [63]

Surgical Innovation

August 2014, USA

Video (Explorer)

n = 27

Clin. study

Lap. ablation

- Manual registration

- Comparison LUS vs. IGS ablation

probe placement.

- Navigation accuracy ¼ 19.56

mm.

- Comparable accuracy IGS vs. LUS

(13.15 mm).

- Additional error

introduced by optical

tracking of flexible

ablation probe.

Sindram et al. [52]

HPB

January 2015, USA

LUS

n = 13

Exploratory

Lap. ablation

- EM tracked LUS.

- Ablation probe position and needle

trajectory visualised.

- Clinical evaluation.

34 lesions ablated in 13 patients.

- Incomplete ablation n = 1.

- Re-ablation in 7 % (same sitting).

- Clin. Outcomes: complications n

= 3; no mortality.

- No accuracy data.

- No data on early

recurrence.

- No control group.

Pessaux et al. [27]

Langenbecks Archives

of Surgery

April 2015, France

Video-AR

n = 3

Exploratory

Robotic

- Manual registration.

- One-off deformation to adjust to

pneumoperitoneum.

- External beam projection of 3D

model.

- AR aided in the identification of

tumour and other structures.

- No accuracy data.

- No surgeon feedback.

Haouchine et al. [43]

IEEE Trans Vis Comput

Graph

May 2015, France

Video-AR

n = 1

Exploratory

Laparoscopy

- Semi-automatic registration with

SSR.

- Deformable, biochemical 3D

model.

- Individual deformation modelling

for liver parenchym and blood

vessels.

- Navigation accuracy app. 4 mm.

- Frame rate of 25 fps.

- Increasing number of 3D model

elements improves accuracy.

- Retrospective

registration.

- Functionality depends on

good initial manual

registration.

- One subject only.

Murakami et al. [58]

Surgery Today

September 2015,

Japan

MRI

n = 6

Exploratory

Lap. ablation

- Designed MR-compatible, weakly

ferromagnetic laparoscope.

- Clinical feasibility demonstrated.

- No significant complications.

- Mean procedure time 275 min.

- Long procedure time.

- No control group.

Plantef`eve et al. [24]

Annals of Biomedical

Engineering

January 2016, France

Video-AR

n = 2

Exploratory

Laparoscopy

- Deformable, biomechanical 3D

model.

- Individual deformation modelling

of parenchym, blood vessels and

Glissonian capsule.

- Landmarks used in addition to

surface registration.

- Navigation accuracy < 1.1 mm.

- Only feasible if 3040 % of liver

surface is visible.

- Use of landmarks creates

deformation boundaries that

improves registration and 3D

modelling.

- Retrospective

registration.

- Further development

from Ref. [43] but

reports new findings.

Huber et al. [64]

Zeitschrift für

Gastroenterologie

January 2016,

Germany

Video (CAS-One

Surgery) n = 1

Case report

LLR

- Manual registration.

- 3D model based on CT prior to

neoadjuvant chemotherapy

- Vanished liver lesion (i.e. not

visible on LUS or inspection)

localised by IGS.

- No accuracy data.

- One subject only.

Schneider et al. [65]

HPB

April 2016, UK

Video-AR

(SmartLiver) n =

11

Exploratory

Lap. and LLR

- Manual and semi-automatic regis-

tration with SSR.

- Evaluation of usability.

- Structured surgeon feedback.

- Setup time app. 21 min.

- Feedback suggests the setup

process is too complex.

- No accuracy data

- Part retrospective

analysis.

Conrad et al. [66]

Journal of the

American College of

Video-AR (CAS-

One Surgery) n

= 1

Case report

LLR

- Manual registration.

- Two-stage hepatectomy.

- Navigation accuracy app. 5 mm.

- One subject only.

- Not compared to LUS.

(continued on next page)

C. Schneider et al.

Surgical Oncology 38 (2021) 101637

9

Table 2 (continued)

Author &

Journal &

Date and Country

Imaging modality

& No. of subjects

Study design

& Type of

surgery

Methodology

Important findings

Important limitations

Surgeons

October 2016, USA

- AR used to guide liver transection

during 1st stage.

- IGS useful for orientation but is

unable to identify all relevant

anatomical structures.

- Registration time app. 1 min.

Aoki et al. [51]

The American Surgeon

December 2016, Japan

LUS

n = 1

Exploratory

LLR

- EM tracked LUS.

- Co-registration of LUS and CT scan.

- Intrahepatic structures manually

highlighted on CT.

- Able to visualise spatial

relationship between surgical

instruments and anatomical

structures.

- No accuracy data.

- IGS technology not

described.

- One subject only.

Robu et al. [29]

IJCARS

July 2017, UK

Video-AR

(SmartLiver) n = 1

Exploratory

LLR

- Semi-automatic registration with

SSR.

- Systematic scoring to evaluate

optimal laparoscope positions for

facilitating SSR.

- Navigation accuracy ¼ 4.7 mm

- Method improved TRE from 17.5

mm to 4.7 mm.

- Identified 4 optimal surface

patches for registration

- Further development

from Ref. [65] but

reports new findings.

- One subject only.

Tinguely et al. [67]

Surgical Endoscopy

October 2017,

Switzerland

Video (CAS-One

Surgery) n = 51

Clin. study

Lap. ablation

- Manual registration.

- IGS guided liver ablation.

- Evaluation of IGS performance and

clinical outcomes.

- Navigation accuracy ¼ 8.1 mm.

- Successful registration in all

patients.

- Calibration time = 1 min;

Registration time = 4 min.

- Early recurrence n = 16.

- No control group.

- Concomitant bowel or

liver resection in some

patients.

Phutane et al. [68]

Surgical Endoscopy

January 2018, France

Video-AR

n = 1

Video pres.

LLR

- Manual registration.

- Empiric evaluation during major

hepatectomy.

- AR aided identification of

transection plane, middle hepatic

vein and tumour.

- AR less useful during transection

due to organ deformation.

- No accuracy data.

- Only one case described

although 8 cases

performed.

- IGS technology not

described.

Heiselman et al. [49]

Journal of Medical

Imaging

April 2018, USA

Video (Explorer)

n = 25

Exploratory

Laparoscopy

- Manual registration.

- Deformable, biomechanical 3D

model.

- Liver ligaments and posterior liver

used as fixed points around which

liver deformation is modelled.

- Comparison of deformable and

rigid 3D modelling.

- Navigation accuracy ¼ 14.7 mm

(rigid model) vs. 7.9 mm (Rucker

method) vs. 6.4 mm (deformable

3D model).

- Registration time 140320s.

- Deformation modelling can be

done preoperatively.

- Frame rate not stated.

- Further development

from Ref. [8] but reports

new findings.

Robu et al. [69]

IJCARS

June 2018, UK

Video-AR

(SmartLiver)

n = 1

Exploratory

LLR

- Semi-automatic registration with

SSR.

- Two step ICP matching

- 1st step coarse registration to

landmark.

- 2nd step fine tuning registration by

SSR.

- Feasibility of 2 step registration

demonstrated.

- Method may form basis for fully

automatic registration without

initial manual alignment.

- No accuracy data.

- Further development

from Ref. [65] but

reports new results.

- One subject only.

Thompson et al. [70]

IJCARS

June 2018, UK

Video-AR

(SmartLiver)

n = 9

Exploratory

Laparoscopy

and LLR

- Manual registration.

- Real-time visual feedback on

navigation accuracy.

- Assessing correlation between

surface landmarks, intrahepatic

structures and navigation accuracy.

- Navigation accuracy app. 12

mm.

- Surface landmarks are reliable

predictors of TRE and suitable

substitutes for intrahepatic

structure localisation.

- Mixed real-time and

retrospective

registration.

- Further development

from Ref. [25] but

reports new results.

Mahmoud et al. [41]

IEEE Trans Med

Imaging July 2018,

France

Video-AR

n = 1

Exploratory

Laparoscopy

- Dense SLAM for registration and

tracking.

- IGS works with monocular

laparoscopes.

- Clinical feasibility of SLAM

demonstrated.

- IGS can adapt to minor

deformation (e.g. respiratory

motion).

- Retrospective

registration.

- No in vivo accuracy data.

- One subject only.

Beerman et al. [71]

European journal of

radiology open

December 2018,

Sweden

Video (CAS-One

Surgery) n = not

stated

Clin. study

Lap. ablation

- Manual registration.

- Retrospective analysis of IGS

ablation.

- High frequency jet ventilation

reduces undesired respiratory liver

motion.

- IGS improved user confidence

compared to LUS guidance.

- No accuracy data.

- Number of laparoscopic

cases not stated.

- No control group.

Le Roy et al. [72]

J. of Visceral Surgery

February 2019, France

Video-AR

n = 1

Video pres.

LLR

- Semi-automatic registration.

- One-off deformation to adjust 3D

model to intraoperative in vivo liver

shape.

- IGS localised liver lesion which was

not visible on LUS due to artefact.

- Standard monocular laparoscope

used.

- No accuracy data.

- IGS technology not

described.

- One subject only.

Yasuda et al. [73]

Asian Journal of

Endoscopic Surgery

April 2019, Japan

Video-AR

n = 4

Clin. study

LLR

- Manual registration.

- CT cholangiography incorporated

into 3D model.

- Landmarks measured with tape and

marked with diathermy.

- IGS performance compared LLR vs.

open surgery.

- Navigation accuracy ¼ 8.8 mm

(LLR) vs. 7.5 mm (open), (p ¼

0.68).

- Repeat registration improved

deformation error.

- Surgically exposed liver vessels

used as landmarks.

- Adding more landmarks did not

improve accuracy.

- Registration time 2 min.

- Accuracy not stated for

individual patients.

- Not clear how additional

landmarks were

registered.

Pfeiffer et al. [44]

IJCARS

April 2019, Germany

Video

n = 1

Exploratory

Laparoscopy

- Deformable, data driven 3D model

based on a convolutional neural

network.

- IGS has potential to adapt

deformation to patient specific

factors (e.g. liver consistency).

- No in vivo accuracy data.

- Retrospective

registration.

- One subject only.

(continued on next page)

C. Schneider et al.

Surgical Oncology 38 (2021) 101637

10

popular solution for semi-automatic registration. Semi-automatic

registration could be expanded to cheaper monocular laparoscopes if

registration through shading and motion or SLAM becomes feasible in

the future [33,41,44,80]. CNN have been successfully used to estimate

position and orientation of objects in a 2D image. At 5094 frames per

second this method is faster and more accurate than biomechanical

approaches [81]. Since no 3D laparoscopes are required, CNN could

potentially facilitate ICP tracking and semi-automatic registration in

conjunction with monocular laparoscopes.

There are two main applications for LUS-IGS. Firstly it can be

employed as a registration tool to identify subsurface liver structures (e.

g. vessels) which are subsequently registered to a 3D model or CT scan

[51,53]. Secondly it can facilitate integration of LUS images into an AR

display [36,51,53]. Advantages of LUS are wide availability, portability,

low costs, high image acquisition speed and an excellent resolution and

depth penetration. Disadvantages are its inherent 2D nature and user

dependent accuracy. Co-registration of LUS and CT images as standalone

visualisation may offer some advantages over routine LUS but in our

opinion this is unlikely to provide the same benefit as AR with a 3D

model.

There were only three eligible articles on CT-IGS. Two articles

demonstrated CBCT based registration [54,56] whereas the third article

purported low dose spiral CT as a feasible alternative to CBCT [55].

CT-IGS offer good navigation accuracy, visualisation of intrahepatic

structures and the ability to generate volumetric rather than just surface

data. Disadvantages are low resolution (CBCT), ionising radiation, high

costs and lack of portability [56,82]. At this stage, CT-IGS have the best

published navigation accuracy [55,56] which may make them useful as

a benchmarking tool.

Only two publications reported on MRI-IGS, one on liver resection

and liver ablation, respectively. Advantages of this modality are excel-

lent imaging quality and the ability to generate volumetric data.

Table 2 (continued)

Author &

Journal &

Date and Country

Imaging modality

& No. of subjects

Study design

& Type of

surgery

Methodology

Important findings

Important limitations

- Model trained by synthetic data

using multiple organ like meshes.

- Data driven modelling runs at 50

fps.

- No deformation modelling of

surgical manipulation.

Prevost et al. [74]

Journal of

gastrointestinal

surgery

September 2019,

Switzerland

Video-AR (CAS-

One AR)

n = 10

Clinical study

LLR

- Manual registration.

- Further development from

Ref. [17].

- AR overlayed onto 3D video.

- Hepato-caval confluence and porta

hepatis used as preferred

landmarks due to stable position.

- Navigation accuracy ¼ 9.2 mm.

- Selective visualisation of area of

interest.

- Calibration time 43s; Registration

time 8.50 min.

- IGS aids in localising difficult to

identify liver lesions but lacks

precision to fully navigate

resection.

- Not stated how TRE was

calculated in 3D video

space.

Schneider et al. [75]

Surgical Endoscopy

July 2020, UK

Video-AR

(SmartLiver)

n = 18

Clin. study

LLR

- Semi-automatic registration with

SSR

- Comparison of navigation accuracy

manual vs semi-automatic

registration.

- Training of CNN to recognise liver

surface on video.

- Surgeon feedback forms.

- Navigation accuracy ¼ 10.9 mm

(manual) vs. 13.9 mm (semi-

automatic) (p ¼ 0.158)

- Registration successful in n = 16/

18.

- Automatic liver segmentation using

CNN.

- Setup time (1015 min) needs

improvement.

- Mixed real-time and

retrospective

registration.

- Further development

from Ref. [70] but

reports new results.

Zhang et al. [42]

Surgical Endoscopy

August 2020, China

Video-AR

n = 64 (30 IGS vs.

34 no IGS)

Clin. study

LLR

- SLAM for surface reconstruction

and tracking.

- Semi-automatic registration with

SLAM.

- Simultaneous visualisation of AR

and near infrared imaging with

ICG.

- Clinical outcome comparison IGS

vs. no IGS.

- Reduced length of stay and blood

loss in IGS group.

- IGS visualisation of tumour margin

27/30.

- IGS aided in identifying

intrahepatic structures and liver

transection line.

- Setup time 30s.

- No accuracy data.

- IGS technology not

described.

Aoki et al. [76]

Journal of

Gastrointestinal

Surgery

September 2020,

Japan

LUS

n = 27

Clin. study

LLR

- EM tracked LUS to CT registration.

- Anatomical colour coding of

structures in CT.

- Navigation accuracy ¼ 12 mm.

- Successful image guidance in 26/

27 cases.

- IGS identified 3 lesions not visible

on LUS.

- Registration time <2min; -Setup

time 7min.

- Patient needs to remain

in neutral table position.

- 3D model available but

not registered to patient.

Bertrand et al. [48]

Surgical Endoscopy

December 2020,

France

Video-AR

(Hepataug) n = 17

Clin. study

LLR

- Deformable, biomechanical 3D

model.

- Semi-automatic registration.

- Further development from

Ref. [72].

- No interruption to workflow

- Good correlation between LUS and

IGS

- Two lesions identified that were

not visible on LUS.

- No data on accuracy or

workflow interruption.

- IGS technology not

described.

Aoki et al. [77]

Surgical Oncology

December 2020, Japan

Video-AR

n = 1

Case report

LLR

- Manual registration.

- AR-guided needle puncture of

portal vein branch.

- Positive ICG staining technique of

liver segments.

- Headset visualisation.

- Portal vein branch accurately

targeted.

- Operative time 285 min.

- No accuracy data

- Registered 3D model

available but not utilised

- Very long procedure

time.

Table 2. Summary of included clinical articles. Published navigation accuracy data is highlighted in bold. Journal abbreviations: IJCARS - International Journal of

Computer Assisted Radiology and Surgery; J Hepatobil Pancreat Sci - Journal of Hepato-Biliary-Pancreatic Sciences; J Surg Res - Journal of Surgical Research;

Hepatology Int. - Hepatology International; J. of Visceral Surgery Journal of Visceral Surgery; IEEE Trans Vis Comput Graph - IEEE Transactions on Visualisation and

Computer Graphics; IEEE Trans Med Imaging - IEEE Transactions on Medical Imaging.

C. Schneider et al.

Surgical Oncology 38 (2021) 101637

11

Disadvantages are incompatibility with standard surgical equipment,

long image acquisition time, very high costs and limited availability.

Surgical freedom of movement is restricted by the size and shape of the

MRI scanner (Fig. 5).

Four articles, all based on Video-IGS, investigated IGS in robotic

assisted surgery [17,24,27,43]. The feasibility of translating IGS meth-

odology from a laparoscopic [27] or open [17] setting to robotic assisted

surgery has been demonstrated. Compared to robotic assisted surgery,

laparoscopic surgery is more widely disseminated and cheaper [83,84].

Therefore it is probable that most IGS innovations will be developed for

LLR initially and subsequentially transferred to a robotic platform if

clinical benefit is sufficiently incentivising.

A number of limitations have to be taken into account. A meta-

analysis of navigation accuracy would have been useful but since a

variety of TRE calculation methods is used by different groups this was

technically not possible. Because this review exclusively focused on in

vivo studies it is possible that recent developments that were only

evaluated ex vivo are not included. In our experience however the

translation process from ex vivo to clinically relevant IGS research is long

and we found that many ex vivo studies have limited surgical relevance.

In conclusion it is the authors opinion that due to aforementioned

advantages Video and LUS -IGS have the best potential to be developed

into useful tools for LLR. The navigation accuracy of CT-IGS is user in-

dependent and hence it may prove valuable as a benchmark control for

new IGS technology. A generalised summary for practical considerations

of different IGS modalities is shown in Table 3.

Current IGS technology requires further advances to evolve into a

fully dependable navigation tool [42,64]. To allow effective comparison

Fig. 6. Graphic showing published navigation ac-

curacy of Video-IGS which demonstrates that

reporting of navigation accuracy is becoming

increasingly common. Although different evalua-

tion methods are used there appears to be less

discrepancy between the results of different groups

in recent years. Studies where no intraoperative

registration was carried out have been excluded. If

accuracy values between different groups were

compared then only the best value is stated. *Study

with only one subject.

Fig. 7. Different methods of enhanced

rendering are showcased on the same video

sequence showing the right liver with over-

layed hepatic veins (purple), portal veins

(blue), hepatic arteries (red), liver tumours

(green) and gallbladder (yellow). a) Plane

clipping can show what is inside a structure

arrow pointing out hepatic vein branch

draining the tumour (purple with green hazy

outline) b) Distance fogging enhances

perception of distance by shading objects

differently arrow pointing at a segmental portal vein branch whose greater transparency indicates an increased distance from the surgeons viewpoint c) Tradi-

tionally anatomical structures are shown completely filled with colour which makes it impossible to see what is behind a structure. Shape outlining enhances edges

that surround structures to improve 3D scene perception and interpretation arrow indicating border between tumour and gallbladder. (original images by Ref. [70]

licensed under CC-BY 4.0). . (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

Table 3

Characteristics of different IGS modalities.

IGS modality

Navigation accuracy

Availability

Transportability

Costs

Main limitation

Video

+

+++

+++

+

Rigid 3D model

LUS

+

+++

+++

+

2D imaging

CT

++

++

+

++

Ionising radiation exposure &

Rigid 3D model

MRI

+++(#)

+

+

+++

Incompatibility with surgical instruments

Table 3. Shown are practical considerations for each IGS modality discussed in this article. # Navigation accuracy not stated but in principle MRI images visualise the

actual intraoperative situation and therefore account for organ deformation and movement.

C. Schneider et al.

Surgical Oncology 38 (2021) 101637

12

of clinical benefits a standardised approach in the evaluation of navi-

gation accuracy would be beneficial [46,70]. An essential step to facil-

itate this is to encourage interdisciplinary collaboration between

imaging scientists and hepatobiliary surgeons and it is hoped that this

review will contribute to this process.

Funder statement

This publication presents independent research commissioned by the

Health Innovation Challenge Fund (HICF-T4-317), a parallel funding

partnership between the Wellcome Trust and the Department of Health.

The views expressed in this publication are those of the author(s) and

not necessarily those of the Wellcome Trust or the Department of Health.

In addition this work was supported by the Wellcome/EPSRC

[203145Z/16/Z].

Declaration of competing interest

Professor Hawkes is a co-founder of IXICO Ltd. Drs. Schneider and

Allam as well as Profs. Davidson, Gurusamy and Stoyanov have no

conflict of interest to declare.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.

org/10.1016/j.suronc.2021.101637.

Author statement

Crispin Schneider: Data curation, Writing manuscript; Moustafa

Allam: Data curation, Revision of manuscript; Danail Stoyanov: Com-

puter science expertise, Review & Editing; Kurinchi Gurusamy: Sys-

tematic review expertise, Methodology; David Hawkes: Medical physics

expertise, Validation, Funding; Brian Davidson: Conceptualization, Su-

pervision, Funding.

References

[1] R. Ciria, D. Cherqui, D.A. Geller, J. Briceno, G. Wakabayashi, Comparative short-

term benefits of laparoscopic liver resection: 9000 cases and climbing, Ann. Surg.

263 (4) (2016 Apr) 761777.

[2] D. Fuks, F. Cauchy, S. Ft´eriche, T. Nomi, L. Schwarz, S. Dokmak, et al., Laparoscopy

decreases pulmonary complications in patients undergoing major liver resection,

Ann. Surg. (2015) 1.

[3] G. Wakabayashi, D. Cherqui, D.A. Geller, J.F. Buell, H. Kaneko, H.S. Han, et al.,

Recommendations for laparoscopic liver resection, Ann. Surg. 261 (4) (2015)

619629.

[4] A. El-Gendi, M. El-Shafei, S. El-Gendi, A. Shawky, Laparoscopic versus open

hepatic resection for solitary hepatocellular carcinoma less than 5 cm in cirrhotic

patients: a randomized controlled study, J. Laparoendosc. Adv. Surg. Tech. 28 (3)

(2018 Mar) 302310.

[5] Å.A. Fretland, V.J. Dagenborg, G.M.W. Bjørnelv, A.M. Kazaryan, R. Kristiansen, M.

W. Fagerland, et al., Laparoscopic versus open resection for colorectal liver

metastases, Ann. Surg. 267 (2) (2017) 1.

[6] I. Dagher, N. ORourke, D.A. Geller, D. Cherqui, G. Belli, T.C. Gamblin, et al.,

Laparoscopic major hepatectomy: an evolution in standard of care, Ann Surg 250

(5) (2009) 856860, 2009/10/07.

[7] J. Kirchberg, C. Reißfelder, J. Weitz, M. Koch, Laparoscopic surgery of liver tumors,

Langenbecks Arch. Surg. 398 (2013) 931938.

[8] T.P. Kingham, S. Jayaraman, L.W. Clements, M.A. Scherer, J.D. Stefansic, W.

R. Jarnagin, et al., Evolution of image-guided liver surgery: transition from open to

laparoscopic procedures, J. Gastrointest. Surg. 17 (7) (2013 Jul) 12741282.

[9] X. Cai, Z. Li, Y. Zhang, H. Yu, X. Liang, R. Jin, et al., Laparoscopic liver resection

and the learning curve: a 14-year, single-center experience, Surg. Endosc. (2014)

18.

[10] F. Cauchy, D. Fuks, T. Nomi, L. Schwarz, L. Barbier, S. Dokmak, et al., Risk factors

and consequences of conversion in laparoscopic major liver resection, Br. J. Surg.

102 (7) (2015 Jun 1) 785795.

[11] T. Nomi, D. Fuks, Y. Kawaguchi, F. Mal, Y. Nakajima, B. Gayet, Learning curve for

laparoscopic major hepatectomy, Br. J. Surg. 102 (7) (2015 Jun) 796804.

[12] T. Ishizawa, A.A. Gumbs, N. Kokudo, B. Gayet, Laparoscopic segmentectomy of the

liver: from segment I to VIII, Ann Surg 256 (6) (2012 Dec) 959964.

[13] M. Abu Hilal, F. Di Fabio, M. Abu Salameh, N.W. Pearce, Oncological efficiency

analysis of laparoscopic liver resection for primary and metastatic cancer: a single-

center UK experience, Arch. Surg. 147 (1) (2012 Jan) 4248.

[14] K.T. Nguyen, T.C. Gamblin, D.A. Geller, World review of laparoscopic liver

resection-2,804 patients, Ann Surg 250 (5) (2009 Nov) 831841, 2009/10/06.

[15] H. Topal, J. Tiek, R. Aerts, B. Topal, Outcome of laparoscopic major liver resection

for colorectal metastases, Surg. Endosc. 26 (9) (2012) 24512455.

[16] T.P. Kingham, M.A. Scherer, B.W. Neese, L.W. Clements, J.D. Stefansic, W.

R. Jarnagin, Image-guided liver surgery: intraoperative projection of computed

tomography images utilizing tracked ultrasound, HPB (Oxford) 14 (9) (2012 Sep)

594603.

[17] N.C. Buchs, F. Volonte, F. Pugin, C. Toso, M. Fusaglia, K. Gavaghan, et al.,

Augmented environments for the targeting of hepatic lesions during image-guided

robotic liver surgery, J. Surg. Res. 184 (2) (2013 Oct) 825831.

[18] Q.R.J.G. Tummers, F.P.R. Verbeek, J.M. Prevoo H a, A.E. Braat, C.I.M. Baeten, J.

V. Frangioni, et al., First experience on laparoscopic near-infrared fluorescence

imaging of hepatic uveal melanoma metastases using indocyanine green, Surg.

Innovat. 22 (1) (2014 Feb) 2025.

[19] L. Vigano, A. Ferrero, M. Amisano, N. Russolillo, L. Capussotti, Comparison of

laparoscopic and open intraoperative ultrasonography for staging liver tumours, Br

J Surg 100 (4) (2013) 535542, 2013/01/23.

[20] R. Montalti, G. Berardi, A. Patriti, M. Vivarelli, R.I. Troisi, Outcomes of robotic vs

laparoscopic hepatectomy: a systematic review and meta-analysis, World J.

Gastroenterol. 21 (27) (2015 Jul) 84418451.

[21] N.C. Buchs, F. Volonte, F. Pugin, C. Toso, P. Morel, Three-dimensional laparoscopy:

a step toward advanced surgical navigation, Surg Endosc Other Interv Tech 27 (2)

(2013) 692693.

[22] D.E. Azagury, M.M. Dua, J.C. Barrese, J.M. Henderson, N.C. Buchs, F. Ris, et al.,

Image-guided surgery, Curr. Probl. Surg. 52 (2015) 476520.

[23] T. Okamoto, S. Onda, K. Yanaga, N. Suzuki, A. Hattori, Clinical application of

navigation surgery using augmented reality in hepatobiliary pancreatic surgery,

Surg. Today 45 (4) (2015) 397406.

[24] R. Plantef`eve, I. Peterlik, N. Haouchine, S. Cotin, Patient-specific biomechanical

modeling for guidance during minimally-invasive hepatic surgery, Ann. Biomed.

Eng. 44 (1) (2016 Jan) 139153, 22.

[25] S. Thompson, J. Totz, Y.Y. Song, S. Johnsen, D. Stoyanov, K. Gurusamy, et al.,

Accuracy validation of an image guided laparoscopy system for liver resection,

SPIE Proc (7) (2015) 9415.

[26] S. Satou, T. Mitsui, R. Ninomiya, M. Komagome, N. Akamatsu, F. Ozawa, et al.,

Image overlay navigation of laparoscopic liver resection, Hepatol Int 8 (2014)

1405.

[27] P. Pessaux, M. Diana, L. Soler, T. Piardi, D. Mutter, J. Marescaux, Towards

cybernetic surgery: robotic and augmented reality-assisted liver segmentectomy,

Langenbecks Arch. Surg. 400 (3) (2014 Apr) 381385.

[28] C. Schneider, S. Thompson, M.J. Clarkson, D.J. Hawkes, B.R. Davidson, A novel

approach to image guidance in laparoscopic liver surgery, Surg. Endosc. 20 (2015).

Suppl 1.

[29] M.R. Robu, P. Edwards, J. Ramalhinho, S. Thompson, B. Davidson, D. Hawkes, et

al., Intelligent viewpoint selection for efficient CT to video registration in

laparoscopic liver surgery, Int J Comput Assist Radiol Surg 12 (7) (2017 Jul)

10791088.

[30] M. Fusaglia, M. Peterhans, D. Wallach, G. Beldi, D. Candinas, S. Weber, Validation

of image overlay accuracy in a instrument guidance system for laparoscopic liver

surgery, Hpb (IHPBA 2012) 14 (2012) 520.

[31] M. Hayashibe, N. Suzuki, Y. Nakamura, Laser-scan endoscope system for

intraoperative geometry acquisition and surgical robot safety management, Med.

Image Anal. 10 (4) (2006 Aug) 509519.

[32] D. Stoyanov, Surgical vision, Ann. Biomed. Eng. 40 (2) (2012 Feb) 332345.

[33] D. Reichard, S. Bodenstedt, S. Suwelack, B. Mayer, A. Preukschas, M. Wagner, et

al., Intraoperative on-the-fly organ-mosaicking for laparoscopic surgery, J. Med.

Imaging 2 (4) (2015 Oct 10), 45001.

[34] V.M. Banz, M. Baechtold, S. Weber, M. Peterhans, D. Inderbitzin, D. Candinas,

Computer planned, image-guided combined resection and ablation for bilobar

colorectal liver metastases, World J. Gastroenterol. 20 (40) (2014 Oct)

1499214996, 28.

[35] F. Volont´e, F. Pugin, P. Bucher, M. Sugimoto, O. Ratib, P. Morel, Augmented reality

and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not

only a matter of fashion, J Hepatobiliary Pancreat Sci. 18 (4) (2011 Jul) 506509.

[36] X. Kang, M. Azizian, E. Wilson, K. Wu, A.D. Martin, T.D. Kane, et al., Stereoscopic

augmented reality for laparoscopic surgery, Surg. Endosc. 28 (7) (2014 Jul)

22272235.

[37] S. Nicolau, L. Soler, D. Mutter, J. Marescaux, Augmented reality in laparoscopic

surgical oncology, Surg Oncol 20 (3) (2011) 189201.

[38] A. Teatini, E. Pelanis, D.L. Aghayan, R.P. Kumar, R. Palomar, Å.A. Fretland, et al.,

The effect of intraoperative imaging on surgical navigation for laparoscopic liver

resection surgery, Sci. Rep. 9 (1) (2019 Dec) 18687, 10.

[39] A. Teatini, J. P´erez de Frutos, B. Eigl, E. Pelanis, D.L. Aghayan, M. Lai, et al.,

Influence of sampling accuracy on augmented reality for laparoscopic image-

guided surgery, Minim Invasive Ther. Allied Technol. (2020) 110.

[40] E. Gibson, M.R. Robu, S. Thompson, P.E. Edwards, C. Schneider, K. Gurusamy, et

al., Deep residual networks for automatic segmentation of laparoscopic videos of

the liver, in: R.J. Webster, B. Fei (Eds.), Prog Biomed Opt Imaging - Proc SPIE, vol.

10135, 2017 Mar, p. 101351M, 3.

[41] N. Mahmoud, T. Collins, A. Hostettler, L. Soler, C. Doignon, J.M.M. Montiel, Live

tracking and dense reconstruction for hand-held monocular endoscopy, IEEE

Trans. Med. Imag. 38 (1) (2019 Jul) 7989, 13.

C. Schneider et al.

Surgical Oncology 38 (2021) 101637

13

[42] P. Zhang, H. Luo, W. Zhu, J. Yang, N. Zeng, Y. Fan, et al., Real - time navigation for

laparoscopic hepatectomy using image fusion of preoperative 3D surgical plan and

intraoperative indocyanine green fluorescence imaging, Surg. Endosc. 34 (8)

(2020) 34493459.

[43] N. Haouchine, S. Cotin, I. Peterlik, J. Dequidt, E. Kerrien, M. Berger, et al., Impact

of soft tissue heterogeneity on augmented reality for liver surgery, IEEE Trans.

Visual. Comput. Graph. 21 (5) (2015 May) 584597, 1.

[44] M. Pfeiffer, C. Riediger, J. Weitz, S. Speidel, Learning soft tissue behavior of organs

for surgical navigation with convolutional neural networks, Int J Comput Assist

Radiol Surg (2019 Apr) 19, 16.

[45] J.S. Heiselman, L.W. Clements, J.A. Collins, J.A. Weis, A.L. Simpson, S.

K. Geevarghese, et al., Characterization and correction of intraoperative soft tissue

deformation in image-guided laparoscopic liver surgery, J. Med. Imaging (2) (2018

Apr) 5, 021203.

[46] R. Modrzejewski, T. Collins, B. Seeliger, A. Bartoli, A. Hostettler, J. Marescaux, An

in vivo porcine dataset and evaluation methodology to measure soft-body

laparoscopic liver registration accuracy with an extended algorithm that handles

collisions, Int J Comput Assist Radiol Surg 14 (7) (2019 May) 12371245, https://

doi.org/10.1007/s11548-019-02001-4.

[47] D. Reichard, D. H¨antsch, S. Bodenstedt, S. Suwelack, M. Wagner, H. Kenngott, et

al., Projective biomechanical depth matching for soft tissue registration in

laparoscopic surgery, Int J Comput Assist Radiol Surg 12 (7) (2017) 11011110.

[48] L.R. Bertrand, M. Abdallah, Y. Espinel, L. Calvet, B. Pereira, E. Ozgur, et al., A case

series study of augmented reality in laparoscopic liver resection with a deformable

preoperative model, Surg. Endosc. 34 (12) (2020) 56425648.

[49] J.S. Heiselman, L.W. Clements, J.A. Collins, J.A. Weis, A.L. Simpson, S.

K. Geevarghese, et al., Characterization and correction of intraoperative soft tissue

deformation in image-guided laparoscopic liver surgery, 14, J Med imaging

(Bellingham, Wash) (2) (2018 Apr) 5, 021203.

[50] L.W. Lau, X. Liu, W. Plishker, K. Sharma, R. Shekhar, T.D. Kane, Laparoscopic liver

resection with augmented reality: a preclinical experience, J. Laparoendosc. Adv.

Surg. Tech. 29 (1) (2019 Jan) 8893.

[51] T. Aoki, M. Murakami, T. Koizumi, A. Fujimori, Y. Enami, T. Kusano, et al.,

Ultrasound with electromagnetic tracking navigation and image fusion system in

laparoscopic liver surgery: an initial clinical experience, Am. Surg. 82 (12) (2016

Dec) e366368, 1.

[52] D. Sindram, K.A. Simo, R.Z. Swan, S. Razzaque, D.J. Niemeyer, R.M. Seshadri, et

al., Laparoscopic microwave ablation of human liver tumours using a novel three-

dimensional magnetic guidance system, HPB 17 (1) (2015 Jan) 8793.

[53] Y. Song, J. Totz, S. Thompson, S. Johnsen, D. Barratt, C. Schneider, et al., Locally

rigid, vessel-based registration for laparoscopic liver surgery, Int J Comput Assist

Radiol Surg 10 (12) (2015 Dec) 19511961.

[54] M. Feuerstein, T. Mussack, S.M. Heining, N. Navab, Intraoperative laparoscope

augmentation for port placement and resection planning in minimally invasive

liver resection, IEEE Trans. Med. Imag. 27 (3) (2008 Mar) 355369.

[55] R. Shekhar, O. Dandekar, V. Bhat, M. Philip, P. Lei, C. Godinez, et al., Live

augmented reality: a new visualization method for laparoscopic surgery using

continuous volumetric computed tomography, Surg Endosc Other Interv Tech 24

(8) (2010 Aug) 19761985.

[56] H.G. Kenngott, M. Wagner, M. Gondan, F. Nickel, M. Nolden, A. Fetzer, et al., Real-

time image guidance in laparoscopic liver surgery: first clinical experience with a

guidance system based on intraoperative CT imaging, Surg. Endosc. 28 (3) (2014

Mar) 933940.

[57] S.S. Chopra, J. Rump, S.C. Schmidt, F. Streitparth, C. Seebauer, G. Schumacher, et

al., Imaging sequences for intraoperative MR-guided laparoscopic liver resection in

1.0-T high field open MRI, Eur. Radiol. 19 (9) (2009 Sep) 21912196.

[58] K. Murakami, S. Naka, H. Shiomi, H. Akabori, Y. Kurumi, S. Morikawa, et al., Initial

experiences with MR Image-guided laparoscopic microwave coagulation therapy

for hepatic tumors, Surg. Today 45 (9) (2015 Sep) 11731178.

[59] K. Konishi, M. Nakamoto, Y. Kakeji, K. Tanoue, H. Kawanaka, S. Yamaguchi, et al.,

A real-time navigation system for laparoscopic surgery based on three-dimensional

ultrasound using magneto-optic hybrid tracking configuration, Int J Comput Assist

Radiol Surg 2 (1) (2007) 110.

[60] J. Ramalhinho, M.R. Robu, S. Thompson, K. Gurusamy, B. Davidson, D. Hawkes, et

al., A pre-operative planning framework for global registration of laparoscopic

ultrasound to CT images, Int J Comput Assist Radiol Surg 13 (8) (2018 Aug)

11771186, 2.

[61] H. Luo, D. Yin, S. Zhang, D. Xiao, B. He, F. Meng, et al., Augmented reality

navigation for liver resection with a stereoscopic laparoscope, Comput. Methods

Progr. Biomed. (2019 Oct) 105099, 7.

[62] X. Liu, W. Plishker, T.D. Kane, D.A. Geller, L.W. Lau, J. Tashiro, et al., Preclinical

evaluation of ultrasound-augmented needle navigation for laparoscopic liver

ablation, Int J Comput Assist Radiol Surg 15 (5) (2020) 803810.

[63] C.W. Hammill, L.W. Clements, J.D. Stefansic, R.F. Wolf, P.D. Hansen, D.A. Gerber,

Evaluation of a minimally invasive image-guided surgery system for hepatic

ablation procedures, Surg. Innovat. 21 (4) (2014 Aug) 419426.

[64] T. Huber, J. Baumgart, M. Peterhans, S. Weber, S. Heinrich, H. Lang, et al.,

[Computer-assisted 3D-navigated laparoscopic resection of a vanished colorectal

liver metastasis after chemotherapy], Zeitschrift für Gastroenterol 54 (1) (2016

Jan) 4043.

[65] C. Schneider, S. Thompson, Y. Song, J. Totz, A. Desjardins, K. Gurusamy, et al.,

Preliminary results from a clinical study evaluating a novel image guidance system

for laparoscopic liver surgery, 7, HPB 18 (2017 Jun) e99. Conference Publication.

[66] C. Conrad, M. Fusaglia, M. Peterhans, H. Lu, S. Weber, B. Gayet, Augmented reality

navigation surgery facilitates laparoscopic rescue of failed portal vein

embolization, J. Am. Coll. Surg. 223 (4) (2016) e31e34.

[67] P. Tinguely, M. Fusaglia, J. Freedman, V. Banz, S. Weber, D. Candinas, et al.,

Laparoscopic image-based navigation for microwave ablation of liver tumors a

multi-center study, Surg. Endosc. 31 (10) (2017) 43154324.

[68] P. Phutane, E. Buc, K. Poirot, E. Ozgur, D. Pezet, A. Bartoli, et al., Preliminary trial

of augmented reality performed on a laparoscopic left hepatectomy, Surg Endosc

Other Interv Tech 32 (1) (2018) 514515.

[69] M.R. Robu, J. Ramalhinho, S. Thompson, K. Gurusamy, B. Davidson, D. Hawkes, et

al., Global rigid registration of CT to video in laparoscopic liver surgery, Int J

Comput Assist Radiol Surg 13 (6) (2018 Jun) 947956.

[70] S. Thompson, C. Schneider, M. Bosi, K. Gurusamy, S. Ourselin, B. Davidson, et al.,

In vivo estimation of target registration errors during augmented reality

laparoscopic surgery, Int J Comput Assist Radiol Surg 13 (6) (2018 Jun) 865874.

[71] M. Beermann, J. Lindeberg, J. Engstrand, K. Galm´en, S. Karlgren, D. Stillstr¨om, et

al., 1000 consecutive ablation sessions in the era of computer assisted image

guidance - lessons learned, Eur J Radiol open 6 (2019) 18.

[72] B. Le Roy, E. Ozgur, B. Koo, E. Buc, A. Bartoli, Augmented reality guidance in

laparoscopic hepatectomy with deformable semi-automatic computed tomography

alignment (with video), J. Vis. Surg. 156 (3) (2019 Feb) 261262, https://doi.org/

10.1016/j.jviscsurg.2019.01.009.

[73] J. Yasuda, T. Okamoto, S. Onda, S. Fujioka, K. Yanaga, N. Suzuki, et al.,

Application of image-guided navigation system for laparoscopic hepatobiliary

surgery, Asian J. Endosc. Surg. 13 (1) (2019 Apr) 3945, https://doi.org/10.1111/

ases.12696.

[74] G.A. Prevost, B. Eigl, I. Paolucci, T. Rudolph, M. Peterhans, S. Weber, et al.,

Efficiency, accuracy and clinical applicability of a new image-guided surgery

system in 3D laparoscopic liver surgery, J. Gastrointest. Surg. 24 (10) (2019 Oct)

22512258, https://doi.org/10.1007/s11605-019-04395-7.

[75] C. Schneider, S. Thompson, J. Totz, Y. Song, M. Allam, M.H. Sodergren, et al.,

Comparison of manual and semi-automatic registration in augmented reality

image-guided liver surgery: a clinical feasibility study, Surg. Endosc. 34 (10) (2020

Oct) 47024711, 1.

[76] T. Aoki, D.A. Mansour, T. Koizumi, Y. Wada, Y. Enami, A. Fujimori, et al.,

Laparoscopic liver surgery guided by virtual real-time CT-guided volume

navigation, J. Gastrointest. Surg. (2020) 15.

[77] T. Aoki, T. Koizumi, M. Sugimoto, M. Murakami, Holography-guided percutaneous

puncture technique for selective near-infrared fluorescence-guided laparoscopic

liver resection using mixed-reality wearable spatial computer, Surg Oncol 35

(October) (2020) 476477.

[78] C. Schneider, S. Thompson, K. Gurusamy, M. Clarkson, B. Davidson, Use of

enhanced visualisation methods to decrease the effect of organ motion in image

guided laparoscopic liver surgery, Br. J. Surg. 104 (2017 Jul) 5243. Suppl(ASGBI

Abstracts 2017).

[79] Y. Uchida, K. Taura, M. Nakao, S. Uemoto, A clinical pilot study of Resection

Process Map: a novel virtual hepatectomy software to visualize the resection

process, case series, Int. J. Surg. 71 (2019 Nov) 3640, 1.

[80] L. Maier-Hein, P. Mountney, A. Bartoli, H. Elhawary, D. Elson, Groch a, et al.,

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic

surgery, Med. Image Anal. 17 (8) (2013) 974996.

[81] B. Tekin, S.N. Sinha, P. Fua, Real-time seamless single shot 6D object pose

prediction. In: proceedings of the IEEE computer society conference on computer

vision and pattern recognition, IEEE Computer Society (2018) 292301.

[82] M. Zijlmans, T. Langø, E.F. Hofstad, C.F.P. Van Swol, A. Rethy, Navigated

laparoscopy liver shift and deformation due to pneumoperitoneum in an animal

model, Minim Invasive Ther. Allied Technol. 21 (3) (2012 May) 241248.

[83] V. Packiam, D.L. Bartlett, S. Tohme, S. Reddy, J.W. Marsh, D.A. Geller, et al.,

Minimally invasive liver resection: robotic versus laparoscopic left lateral

sectionectomy, J. Gastrointest. Surg. 16 (12) (2012 Dec) 22332238.

[84] C.M. Ho, G. Wakabayashi, H. Nitta, N. Ito, Y. Hasegawa, T. Takahara, Systematic

review of robotic liver resection, Surg Endosc Other Interv Tech 27 (3) (2013 Mar)

732739.

[85] S.S. Chopra, S.C. Schmidt, R. Eisele, U. Teichgr¨aber, I. Van Der Voort, C. Seebauer,

et al., Initial results of MR-guided liver resection in a high-field open MRI, Surg

Endosc Other Interv Tech 24 (10) (2010) 25062512.

C. Schneider et al.